Skip to content

[WIP]add the confidence interval computation - #387

Open
YazhiGao wants to merge 1 commit into
linkedin:masterfrom
YazhiGao:ratio_random_effect_select
Open

[WIP]add the confidence interval computation#387
YazhiGao wants to merge 1 commit into
linkedin:masterfrom
YazhiGao:ratio_random_effect_select

Conversation

@YazhiGao

@YazhiGao YazhiGao commented Aug 14, 2018

Copy link
Copy Markdown

this is currently the implementation of ratio modeling for feature selection of random effect in photon.
I follow the algorithm described in the original publication but some twists are made according to discussion with yiming and alex.

  • Unit tests all pass.

  • Integration tests all pass.

The algorithm in reality(highly related with codebase instead of only mathematical expression) is as follows:

1.pass in the featureStatisticSummary
2.identify the binomial columns
3.compute the lowerbound for binomial columns based on the t value
4.select the feature based on only the following lowerbound criterion(non-binomial and intercept columns are kept automatically)

if (t < 1) {
  T_l = 1 / T_u
}

if (T_l > 1D) {
  //  select feature
}

As a WIP commit, there are things to polish in near future since we currently focus on the feasibility of this experimental method and try to minimize user-side changes :

  • unit tests not fully covering all scenarios of feature selection. Currently the binomial cases are not selected, we need to craft some data that covering all cases.

  • binomial feature column identification predicate needs to be stronger. Current solution is inherently flawed, we need more computation at feature summary stage to ensure this one.

  • hyperparameter interface design. for convenience purposes, the user side interface for pass in normal distribution quartile and lowerbound threshold hyperparameter redesign.

  • the relationship with pearson correlation feature selection. We need another parameter to decide on the algorithm of feature selection or mix them in later stage.

  • crafted test data need some change, currently some unneeded feature summary entries are not carefully addressed.

  • further experiment report and benchmark report after regression tests

  • the way we currently keep non-binary and intercept columns is not good for further feature ranking report planned. need redesign

@joshvfleming @ashelkovnykov

Loading
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

7 participants